Retrospective Evaluation of Various Serological Assays and Multiple Parameters for Optimal Diagnosis of Lyme Neuroborreliosis in a Routine Clinical Setting

ABSTRACT Laboratory diagnosis of Lyme neuroborreliosis (LNB) is challenging, and validated diagnostic algorithms are lacking. Therefore, this retrospective cross-sectional study aimed to compare the diagnostic performance of seven commercial antibody assays for LNB diagnosis. Random forest (RF) modeling was conducted to investigate whether the diagnostic performance using the antibody assays could be improved by including several routine cerebrospinal fluid (CSF) parameters (i.e., leukocyte count, total protein, blood-CSF barrier functionality, and intrathecal total antibody synthesis), two-tier serology on serum, the CSF level of the B-cell chemokine (C-X-C motif) ligand 13 (CXCL13), and a Borrelia species PCR on CSF. In total, 156 patients were included who were classified as definite LNB (n = 10), possible LNB (n = 7), or non-LNB patient (n = 139) according to the criteria of the European Federation of Neurological Societies using a consensus strategy for intrathecal Borrelia-specific antibody synthesis. The seven antibody assays showed sensitivities ranging from 47.1% to 100% and specificities ranging from 95.7% to 100%. RF modeling demonstrated that the sensitivities of most antibody assays could be improved by including other parameters to the diagnostic repertoire for diagnosing LNB (range: 94.1% to 100%), although with slightly lower specificities (range: 92.8% to 96.4%). The most important parameters for LNB diagnosis are the detection of intrathecally produced Borrelia-specific antibodies, two-tier serology on serum, CSF-CXCL13, Reibergram classification, and pleocytosis. In conclusion, this study shows that LNB diagnosis is best supported using multiparameter analysis. Furthermore, a collaborative prospective study is proposed to investigate if a standardized diagnostic algorithm can be developed for improved LNB diagnosis. IMPORTANCE The diagnosis of LNB is established by clinical symptoms, pleocytosis, and proof of intrathecal synthesis of Borrelia-specific antibodies. Laboratory diagnosis of LNB is challenging, and validated diagnostic algorithms are lacking. Therefore, this retrospective cross-sectional study aimed to compare the diagnostic performance of seven commercial antibody assays for LNB diagnosis. Multiparameter analysis was conducted to investigate whether the diagnostic performance using the antibody assays could be improved by including several routine (CSF) parameters. The results of this study show that LNB diagnosis is best supported using the detection of intrathecally produced Borrelia-specific antibodies, two-tier serology on serum, CSF-CXCL13, Reibergram classification, and pleocytosis. Furthermore, we propose a collaborative prospective study to investigate the potential role of constructing a diagnostic algorithm using multiparameter analysis for improved LNB diagnosis.

The paper is an impressive result of an extensive analysis of paired CSF-serum samples in various assays for Lyme neuroborreliosis. I have a number of comments. 1. In table 4, details on sensitivity and specificity of the various assays are shown, and it is clear that some patients, especially possible LNB, are negative. As stated by the authors (l.83-88) this could be due to short duration of disease or early treatment. However, this is not discussed in the text. Adding duration of disease -and, if known, early antibiotic treatment to supplementary figure 3 (only relevant for cases, not for controls) and discussing them in the text on sensitivity of antibody assays would greatly add to the value of the manuscript. 2. L. 326: Details on the selection of the 156 patients have already been published (31). The authors refer to reference 31, and one gets the impression that the same panel of patients is described. However, a close look at the number of dLNB, pLNB and non-Lyme cases in this reference show that these differ between the numbers in this manuscript and the numbers in reference 31. The authors should explain these differences. Apparently a few different samples have been used. Alternatively, patients could have been reclassified, but this might have consequences for the analysis in reference 31. 3. The authors state that all patients were eligible if a paired blood and serum sample for Borrelia testing were received, and in addition (L.119-120) that a prerequisite for patients to be included was the availability of at least 1250 µl of CSF and 110 µl of serum. This could well have influenced the results, especially with regard to inclusion of children, from whom usually less CSF is obtained. The authors do not mention anything about this issue. Table 1 suggests that there were few, if any, children included. If pediatric LNB was excluded beforehand, this could be stated. Otherwise, some epidemiological data regarding LNB cases diagnosed in the study period, but excluded for lack of availability of a blood sample or insufficient volume of CSF would be helpful whether these patients differed from the 17 cases included in the study. Minor comments: 4. PCR for Borrelia species was done, but details (reference?) could be mentioned. 5. To obtain a good impression of a quantitative parameter such as age, duration of disease, pleocytosis, glucose and total protein, ranges or interquartile ranges provide more insight than 95% confidence interval. Since the number of patients with LNB is low, interquartile range would not add much and ranges should be provided. This might be done in a supplementary table if the authors want to keep the 95% Cis in table 3 because of statistics. 6. L 454-459: The sensitivities and NPVs of most RF models were higher than the upper limit of the respective 95% CIs obtained using the results of the antibody assays only, except for the C6 and the Enzygnost ELISA (Table 5 and Figure 1A/B). In contrast, the specificities and PPVs of most RF models were comparable with those of the antibody assays only, except for the IDEIA and the Medac ELISA for which the specificities and PPVs obtained using RF modelling were below the lower limit of the respective 95% CIs obtained using the results of the antibody assays only. I would conclude that the consequence would be that in case of a negative result in an antibody assay, other parameters should be taken into account to increase sensitivity; in contrast, if an antibody assay is positive, other parameters should not be taken that much into account (at least not as done in the RF model), since this might decrease specificity and PPV. Is this correct? If so, could this be stated? 7. A request for a Lyme test on CSF does not tell that the patient really had clinical symptoms suggestive of LNB as defined by ENFS. Apparently, this was not checked by the authors, and for convenience reasons it was supposed that if a clinician had made a request for a Lyme test on CSF, these symptoms existed. I would consider this acceptable, but in the various passages where this is addressed (methods, table 3 note c, supplementary figure 1) the text should rather be "Clinical symptoms suggestive of LNB were supposed to be present when a request....." 8. L.534-536: The EFNS recommends to use an AI calculation to proof (prove?) intrathecal synthesis of Borrelia-specific antibodies (3) and this is confirmed in our study. I would omit that conclusion, since no analysis was performed on assays on CSF only -except C6 and blot, for which the conclusion does not hold for C6 (see below). 9. L.537-539: The NPVs of the antibody assays only and those of the RF models showed that a Reiber-based CSF-serum assay is preferred as the respective NPVs were highest. This is not correct, the NPV of the C6-assay is 99.3% (figure 1) and therefore higher than most of the Reiber-based assays. 10. Abstract l 37-38: RF modelling demonstrated that most of the sensitivities of the antibody assays could be improved by including other parameters. The authors probably mean "sensitivity of most antibody assays...." 11. From l.242-249 it can be concluded that "two tier-serology" means C6 EIA in serum followed by immunoblot. However, in table 3 and especially in table 6 -where CSF-serum pairs are tested with other ELISAs-a footnote could be added to clarify this. In clinical practice, if an ELISA would be used to determine whether intrathecal antibodies are present, the same assay would probably be used to screen serum for anti-Borrelia antibodies, but an analysis in which the different ELISAs were considered as screening assays was not done in this study.

Preparing Revision Guidelines
To submit your modified manuscript, log onto the eJP submission site at https://spectrum.msubmit.net/cgi-bin/main.plex. Go to Author Tasks and click the appropriate manuscript title to begin the revision process. The information that you entered when you first submitted the paper will be displayed. Please update the information as necessary. Here are a few examples of required updates that authors must address: • Point-by-point responses to the issues raised by the reviewers in a file named "Response to Reviewers," NOT IN YOUR COVER LETTER. • Upload a compare copy of the manuscript (without figures) as a "Marked-Up Manuscript" file. • Each figure must be uploaded as a separate file, and any multipanel figures must be assembled into one file. For complete guidelines on revision requirements, please see the journal Submission and Review Process requirements at https://journals.asm.org/journal/Spectrum/submission-review-process. Submissions of a paper that does not conform to Microbiology Spectrum guidelines will delay acceptance of your manuscript. " Please return the manuscript within 60 days; if you cannot complete the modification within this time period, please contact me. If you do not wish to modify the manuscript and prefer to submit it to another journal, please notify me of your decision immediately so that the manuscript may be formally withdrawn from consideration by Microbiology Spectrum.
If your manuscript is accepted for publication, you will be contacted separately about payment when the proofs are issued; please follow the instructions in that e-mail. Arrangements for payment must be made before your article is published. For a complete list of Publication Fees, including supplemental material costs, please visit our website.
Corresponding authors may join or renew ASM membership to obtain discounts on publication fees. Need to upgrade your membership level? Please contact Customer Service at Service@asmusa.org.
Thank you for submitting your paper to Microbiology Spectrum.

Response to the reviewers' comments
General comment to the editor from the author: Thank you for giving us the opportunity to submit a revised version of the manuscript 'Retrospective evaluation of various serological assays and multiple parameters for optimal diagnosis of Lyme neuroborreliosis in a routine clinical setting' for publication in 'Microbiology Spectrum'. We appreciate the time and effort that you and the reviewers dedicated to providing feedback on our manuscript and are grateful for the insightful comments. We have incorporated most of the suggestions and believe these have improved our manuscript. Please see below our point-by-point response (in italic and blue text) to the reviewers' comments. All line and page numbers refer to the revised manuscript with tracked changes.

Reviewer #1: (Comments for the Author):
In this manuscript Van Gorkum et al describe their evaluation of several serological assays and additional laboratory assays to diagnose Lyme neuroborreliosis. The methods used in this study are sound and the conclusions are worthwhile. It's a bit unfortunate that not more neuroborreliosis cases could be included in this study, because this might have aided with the comparison of the performance of the serological assays.
Comment author: We thank the reviewer for thoroughly reading this manuscript. Indeed, the number of Lyme neuroborreliosis (LNB) patients in our study is limited, but this is inherent to the cross-sectional study design. A case-control study would have been easier to perform and would have included more LNB patients, but has a risk of introducing bias, since such a study excludes patients who are difficult to diagnose (1). By choosing a cross-sectional study design, which comprised almost three years, the number of LNB patients was in line with the expected number of LNB patients to be included in our hospital during the predefined study period, as was discussed on page 28 (lines 592-597).
I have no further questions or suggestions for this manuscript. I think the word 'proof' in line 520 and 536 should be replaced by 'prove'.

Comment author:
We thank the reviewer for bringing this to our attention and the suggested corrections have been made (page 27; lines 567 and 583).

Reviewer #2 (Comments for the Author):
The paper is an impressive result of an extensive analysis of paired CSF-serum samples in various assays for Lyme neuroborreliosis. I have a number of comments.

Comment author:
We would like to thank the reviewer for thoroughly reading this manuscript and for the thoughtful comments and constructive suggestions and have revised accordingly. We feel that the manuscript is greatly improved as a result.
1. In table 4, details on sensitivity and specificity of the various assays are shown, and it is clear that some patients, especially possible LNB, are negative. As stated by the authors (l.83-88) this could be due to short duration of disease or early treatment. However, this is not discussed in the text. Adding duration of disease -and, if known, early antibiotic treatment to supplementary figure 3 (only relevant for cases, not for controls) and discussing them in the text on sensitivity of antibody assays would greatly add to the value of the manuscript.  Table S3, last three rows of the definite LNB patients]). Two patients who were classified as non-LNB patient in the published manuscript (17) (i.e. absence of pleocytosis and absence of intrathecal Borreliaspecific antibody synthesis using the IDEIA),were classified as possible LNB patient in the current study using the consensus strategy (see Table S3, last two rows of the possible LNB patients). We have now added the following text to the Results section: • page 17, lines 531-354: The number of possible and definite LNB patients in this study slightly differed from our previous study (17)

as intrathecal Borrelia-specific antibody synthesis was based on either a consensus strategy (this study) or on the IDEIA results (previous study).
We believe that the different classification of these patients does not have an effect on the inferences made with regard to the usefulness of CSF-CXCL13 in LNB diagnostics. Furthermore, in the discussion of the published manuscript (17), we have elaborated on the three possible LNB patients who had pleocytosis and a negative IDEIA result and hypothesized that these negative IDEIA results are caused by the lower sensitivity of the IDEIA in the early stages of LNB, as has been reported by others (12,18). We also mention that intrathecal Borrelia-specific antibody synthesis was shown using other CSF-serum assays and refer to the current manuscript. The hypothesis of an early LNB is further strengthened by the IgM and IgG Reibergrams, as these show a disturbed blood-CSF barrier and intrathecal total-IgM synthesis in the absence of intrathecal total-IgG synthesis (17).
3. The authors state that all patients were eligible if a paired blood and serum sample for Borrelia testing were received, and in addition (L.119-120) that a prerequisite for patients to be included was the availability of at least 1250 μl of CSF and 110 μl of serum. This could well have influenced the results, especially with regard to inclusion of children, from whom usually less CSF is obtained. The authors do not mention anything about this issue. Table 1 suggests that there were few, if any, children included. If pediatric LNB was excluded beforehand, this could be stated. Otherwise, some epidemiological data regarding LNB cases diagnosed in the study period, but excluded for lack of availability of a blood sample or insufficient volume of CSF would be helpful whether these patients differed from the 17 cases included in the study.

Comment author:
We understand the concern of the reviewer. In this study, no selection criteria were applied based on age for the consecutively selected patients. This has now been more clearly stated in the Materials and Methods section (page 7, line 119).The six additional LNB patients who were included in this study had taken part in two other studies of our research group (25,26) and due to the age criterion (≥18 years) used in these two studies, these six LNB patients were adult patients. For the CSF and serum of these additional LNB patients, the same inclusion criteria applied as for the CSF and serum of all consecutive patients. This has now been more clearly stated in the Materials and Methods section (page 7, line 134-138).
However, to address the concern of the reviewer, we also investigated the ages among all consecutive patients for whom a CSF and blood sample was drawn less than 24 hours apart, which involved 423 of the 1098 patients (see Figure 1 of our published manuscript (17) 6. L 454-459: The sensitivities and NPVs of most RF models were higher than the upper limit of the respective 95% CIs obtained using the results of the antibody assays only, except for the C6 and the Enzygnost ELISA (Table 5 and Figure 1A/B). In contrast, the specificities and PPVs of most RF models were comparable with those of the antibody assays only, except for the IDEIA and the Medac ELISA for which the specificities and PPVs obtained using RF modelling were below the lower limit of the respective 95% CIs obtained using the results of the antibody assays only. I would conclude that the consequence would be that in case of a negative result in an antibody assay, other parameters should be taken into account to increase sensitivity; in contrast, if an antibody assay is positive, other parameters should not be taken that much into account (at least not as done in the RF model), since this might decrease specificity and PPV. Is this correct? If so, could this be stated? Re: Spectrum00061-22R1 (Retrospective evaluation of various serological assays and multiple parameters for optimal diagnosis of Lyme neuroborreliosis in a routine clinical setting) Dear Dr. Tamara van Gorkom: The authors have thoroughly addressed the concerns of the reviewers.
Your manuscript has been accepted, and I am forwarding it to the ASM Journals Department for publication. You will be notified when your proofs are ready to be viewed.
The ASM Journals program strives for constant improvement in our submission and publication process. Please tell us how we can improve your experience by taking this quick Author Survey.
As an open-access publication, Spectrum receives no financial support from paid subscriptions and depends on authors' prompt payment of publication fees as soon as their articles are accepted. You will be contacted separately about payment when the proofs are issued; please follow the instructions in that e-mail. Arrangements for payment must be made before your article is published. For a complete list of Publication Fees, including supplemental material costs, please visit our website.
ASM policy requires that data be available to the public upon online posting of the article, so please verify all links to sequence records, if present, and make sure that each number retrieves the full record of the data. If a new accession number is not linked or a link is broken, provide production staff with the correct URL for the record. If the accession numbers for new data are not publicly accessible before the expected online posting of the article, publication of your article may be delayed; please contact the ASM production staff immediately with the expected release date.